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Background and Context 

Starting in 2002, the Bill & Melinda Gates Foundation, partnering with other funding agencies, 
created the Early College High School Initiative, which is leading to the widespread creation of 
ECHSs throughout the country. Over 200 ECHSs have been started since 2002. While there is 
little research to support the ECHS design (American Institutes for Research & SRI 
International, 2005; Jacobson, 2005), evaluations in Middle College programs suggest that 
middle colleges can increase the graduation rates and college attendance of low-performing 
students (Cullen, 1991; Houston, Beyers, & Danner, 1992). However, an experimental study of 
the Middle College model as implemented in Portland, Oregon found that the model had no 
impact on graduation or dropout rates (Dynarski, Gleason, Rangarajan, & Wood, 1998). With 
widespread creation of ECHS across the country, and in particular in North Carolina, there is a 
pressing need to understand whether ECHS works and for whom. 

Purpose and Goals of Study 

The purpose of this study is to rigorously examine the implementation and impact of the Early 
College High School model in North Carolina. This study is a first of its kind by utilizing 
longitudinal experimental design to assess impacts on student outcomes. Figure 1 presents a 
conceptual framework showing the Design Principles of the Early College Model and the 
program’s anticipated key intermediate and long-term outcomes. 

The primary goal of the ECHS model is to increase the number of students who graduate from 
high school and who continue on and succeed in college. Therefore, the anticipated long-term 
outcomes for the program include increased high school graduation rates, increased enrollment 
in college, and increased graduation from college. To track progress toward those long-term 
outcomes, the conceptual framework identifies intermediate outcomes that are associated with 
continued enrollment in high school and enrollment in and success in college. These intermediate 
outcomes include student attendance (Lee & Burkam, 2003), higher-level course-taking 
(Adelman, 1999; Lee & Burkam, 2003), student attitudes toward self and school (House, 1993; 
Lan & Lanthier, 2003), student behavior in school (Lee & Burkam, 2003), student aspirations 
toward college, and student achievement. The intermediate and long-term outcomes form the 
basis for the impact questions in the Study of the Efficacy of North Carolina’s Learn and Earn 
Early College High School Model. 

The study has three main goals: 

• Determine the impact of the model on selected student outcomes; 

• Determine the extent to which impacts differ by student characteristics; and 

• Examine the implementation of the model by site and the extent to which specific model 
components are associated with positive impacts by site. 

For the purpose of this paper presentation proposal, we present early findings from the first 
cohort of students in our study. We will be updating our analyses to include the most up-to-date 
data for the SREE conference. Here, we present findings based on implementation and impact of 
the ECHS model on students’ attitudes, behaviors, and engagement based on Student Opinion 
Survey data from one year of data (2008), and impacts on students’ course-taking and course- 
progression patterns in the ninth grade using three years of data (2006-2008). 
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Setting 

Located on the campuses of two- or four- year colleges and universities, ECHSs are expected to 
provide an academically rigorous course of study with the goal of ensuring that all students 
graduate with a high school diploma and two years of university transfer credit or an associate’s 
degree. ECHSs are intended to make a priority of serving students who are typically under- 
represented in the college-going population, such as students who are first generation college- 
going, students from low-income families, those who are members of a minority group and those 
who have met with failure in conventional schools. During the 2007-08 school year, 42 ECHSs 
were open for students in North Carolina. Of those 42, four are partnered with a University of 
North Carolina constituent institution, 37 are partnered with a community college, and one 
school is partnered with a virtual college partner. Currently, there are approximately 70 ECHS in 
North Carolina. 

Subjects 

Twenty one ECHSs are in our study. We have been recruiting ECHSs to participate in the study 
since the 2006-2007 school year. We recruited 2 ECHSs for our pilot study in 2006, with 4, 6, 
and 9 new schools recruited in the 2007, 2008, and 2009 school years. Most of these schools 
agreed to provide multiple cohorts of students; for example, in one school, we will have four 
cohorts of 9 th graders. Therefore, by the end of this study, we anticipate on having approximately 
4,000 students. Table 1 presents an overview of the sampling design by school year. As seen in 
this table, by the end of the study (2010-201 1), we will have three cohorts of 9 th and 10 th graders, 
2 cohorts of 1 1 th graders and a cohort of 12 th graders. 

The sample used to estimate the impact of ECHS on students’ course taking and progression 
patterns comprises 718 ninth grade students who were randomly assigned to the ECHS or control 
group (416 treatments and 302 control) in 6 sites between 2006 and 2008. Table 2 shows the 
baseline characteristics of the full sample as a whole and by treatment/control status. While the 
majority of the students is Caucasian (68%), about 20% is African American and 10% of other 
ethnicities such as Hispanic, Asian, Native American, and multiracial. Approximately half (46%) 
of the sample is considered first-generation college bound, and 44% qualify for free or reduced- 
price lunch. 

We also examined whether baseline characteristics of the treatment and control students were 
balanced, which should be the case for most characteristics because of random assignment. The 
last two columns in Table 2 show the treatment-control difference for each characteristic and the 
p- value corresponding to the test of statistical significance. As seen, the treatment and 
comparison group appear to be statistically equivalent, except for three characteristics (retained 
in the past, and passing math and reading in the eighth grade). 

A sub-set of students in 9 th grade in the spring of 2008 also took the Student Opinion Survey. We 
present impact results from 220 students (171 treatment and 49 control) in 4 schools completing 
their 9 th grade year. 

Intervention: Early College High School 

The ECHS initiative is administered jointly by the North Carolina Department of Public 
Instruction (NCDPI) and the North Carolina New Schools Project (NCNSP), a non-profit school 
development organization established in 2003 by the Office of the Governor and the North 
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Carolina Education Cabinet with support from the Bill & Melinda Gates Foundation. The ECHS 
initiative is designed to improve high schools, to better prepare students for college and career, to 
create a seamless curriculum between high school and college, and to provide work-based 
learning experiences to students. 

In order for students to be able to accomplish the goal of two years of college credit or an 
associates degree by the time they graduate from high school, the ECHS must develop, in 
collaboration with their higher education partner, an aligned, seamless curriculum plan that 
provides the high school and college courses that students need to take to complete both degrees 
and that avoids unnecessary duplication and/or omission of critical content. Each ECHS is 
expected to implement and exhibit a specific set of principles, known as Design Principles, 
developed by the NCNSP. Those Design Principles, as articulated by the NCNSP, are as 
follows: 

• Ensuring that students are ready for college; 

• Instilling powerful teaching and learning in schools; 

• Providing high student/staff personalization; 

• Redefining professionalism; and 

• Implementing a purposeful design (North Carolina New Schools Project, December, 2007). 

Research design 

This study is based on a multi-site randomized field trial. From a pool of eligible students, 

ECHSs enrolled students based on random assignment, and the study compares the students who 
were assigned to the treatment group (ECHS) with students who were assigned to the control 
group (business as usual). We refer to each ECHS and business-as-usual schools that enroll 
control group students as a “site”. Therefore, within each site, students are randomly selected to 
attend an ECHS school or another high school. 1 As schools continue to add new 9 th grade classes 
via random assignment each year, those students are added to the study sample; hence some sites 
have multiple cohorts of students. 

Data collection and analysis 

Data used in impact analyses include: 1) Administrative data, collected by the North Carolina 
Department of Public Instruction (NCDPI), and merged and de-identified by the North Carolina 
Education Research Center (NCERDC) at Duke University; and 2) Student Opinion Survey data, 
administered by the study team, starting in 2008. We administer the survey to a subset of 
students who also have administrative data. Taken together, outcomes include attendance, 
course-taking patterns, aspirations, academic achievement, and school-leaving and dropout 
status. The impacts of ECHS on these outcomes are estimated within an Intent-To-Treat (ITT) 
framework, in which a student’s experimental status as a treatment or control student, rather than 
actual participation in an ECHS, serves as our measure of treatment. We use multivariate binary 
choice models (e.g., logistic regression), hierarchical linear models (HLM, as necessary), and 



i 



Other high schools can include traditional comprehensive high schools, charter schools, magnet schools, or private 
schools. 
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analysis of variance (ANOVA) to analyze the administrative and survey data. In our regression- 
based models, we employ baseline student characteristics and site indicators as covariates. 2 

Data on implementation are gathered by student and staff surveys, and qualitative site visits. The 
student survey (Student Opinion Survey) includes both questions about the student’s school 
experience (instructional activities, support activities, college awareness activities, and 
expectations for students) as well as questions about short-term outcomes such as student-teacher 
relationships, and measures of student engagement. The staff survey asks about implementation 
of the design principles’ indicators. Each participating school receives a site visit once over their 
four-year participation in the study. Each site visit includes a tour of the school, observations in 
two classes, and interviews with the principal, the college liaison, two staff members, two 
college faculty members, and a focus group of students. Implementation analyses will include 
qualitative analysis of the site visits, and conducting psychometric analyses to develop latent 
variables of implementation from the surveys. 

Results 

Impact Findings. As part of its accountability system, students are required to take End-of- 
Course exams following the completion of certain subjects including Algebra 1, English 1, and 
Geometry. Students must pass exams in Algebra 1 and English 1 to graduate. In this study, we 
identify those who have taken the course by whether they have taken the end-of-course exams 
(TAKEUP) and consider those who passed the exam to have progressed (PROGRESS). 

Utilizing administrative data, Table 3 shows the early impact results of ECHS on ninth grade 
course take-up (0 = no, 1 = yes) and course progress (0 = no, 1 = yes). Impact estimates are 
reported in marginal effects, which represent the difference in the average probabilities of the 
treatment and control groups taking-up or progressing in a course. In looking at course taking of 
students in ECHS versus the control group, we found a positive impact of ECHS in math 
courses, such as Algebra I and college-preparatory math courses (a composite that measures 
taking and progressing in Algebra I, Algebra II, and Geometry). Specifically, there is a 14 
percentage point difference in Algebra I take-up between ECHS and control students, and a 10 
percentage point difference in Algebra I progress. With college-preparatory math courses, there 
is an 1 1 percentage point difference that ECHS students would take at least one course, and a 
fourteen percentage point difference that ECHS students would take at least two courses than the 
control group students. There are no statistically significant differences in the English I and 
Geometry take-up and progression patterns of treatment and control students. 

Utilizing survey data to augment outcomes data (performance on standardized statewide 
assessments) with measures of attitudinal and behavioral outcomes, the Student Opinion Survey 
includes 6 outcome subscales: English Language Arts Self-Efficacy (ELASE), Math Self- 
Efficacy (MSE), College Instructor Expectations (CEXP), High School Instructor Expectations 
(HSEXP), Behavior, and Schoolwork Engagement. Due to our smaller sample, we ran 
ANOVAs, though we anticipate regressions to control for the same covariates as in our analysis 
above. Our ANOVA findings include no statistically significant differences for MSE, and a 
small statistically significant effect on ELASE ( F 1 37 , = 6.34, p = .01, partial rf = .02) favoring 
the control group, leading us to conclude that the ECHS has not yet produced a measurable effect 



2 Following Schochet (2008), these regressions do not include a school-level since this is student-level random 
assignment and we will not seek to generalize the results outside our sample. 
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on student perceptions of themselves as learners. Students in ECHSs do perceive higher teacher 
expectations (F 1371 = 19.77, p < .01, partial rf = .05) than did THS students, though these effect 
were small. The remaining constructs were found to have no statistically significant effects or 
detailed interactions that require more space to discuss. 

Implementation Findings. Results show that the ECHS are implementing the design principles 
as intended and that many of these components are experienced at a much higher level by ECHS 
students as compared to students in the traditional high school. Table 4 shows statistically 
significant results for some key implementation scales on the Student Opinion Survey (2008). 
Highlights include higher staff- student relationships and student support activities for ECHS 
students compared to the students in traditional high schools. 

Conclusions 

This study represents a unique opportunity to investigate impacts of the early college high school 
model, a type of high school reform that focuses on intense college preparation particularly for 
first-generation college bound students. Highlighting early findings from both the impact and 
implementation results, we found that schools are implementing the Design Principals as 
intended, where students are reporting higher levels of support and interactions with school staff. 
According to the logic model, this should lead to higher course ECHS students have high 
expectations of college attendance and frequents college facilities. ECHS students also take more 
Algebra courses than the control group, indicating early positive impacts on ECHS. However, we 
are finding limited impacts on student’s engagement, expectations, and attitudes towards school 
work. 

We will update our analyses and results to include the 2009 Student Opinion Survey for the 
implementation and impact analyses, and controlling for the same students covariates and site 
indicators as the analyses using the administrative data. 

Further, while these are early signs of promise in the ECHS model, we will also investigate the 
impacts of ECHS on other outcomes of interest, such as attendance, achievement, and 
motivations. The next phase in our analyses is to utilize the implementation data to understand 
associations between how well ECHS is implemented and how students achieve. Additional 
analyses will include understanding the relationship between implementation of ECHS and 
student outcomes. 
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Appendix B: Figures and Tables 

Figure 1: Conceptual Framework of North Carolina’s Learn and Earn Early College High 
School Model 
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Table 1. Experimental Design Sample by Cohort and Year 





2006-2007 


2007-2008 


2008-2009 


2009-2010 


2010-2011 




Grade 


Students 


Grade 


Students 


Grade 


Students 


Grade 


Students 


Grade 


Students 


Pilot 


9 th 


293 


10 th 


249 


11 th 


211 


12 th 


179 






Cohort 1 






9 th 


369 


10 th 


314 


11 th 


267 


12 th 


227 


Cohort 2 










9 th 


1113 


10 th 


946 


11 th 


804 


Cohort 3 














9 th 


1342 


10 th 


1141 



Notes: Sample assumes a study attrition rate of 15% per school year. There are 21 ECHS schools in the sample, 
most providing multiple cohorts of data. 



Table 2: Descriptive Statistics of the Sample from the Administrative Data 





Whole Sample 
(N=718) 


Treatment Group 
(N=416) 


Control Group 
(N=302) 


T-C Difference 




N 


Mean 


N 


Mean 


N 


Mean 


Difference 


P-Value 


Race & Ethnicity 


















American Indian 


718 


0.56% 


416 


0.96% 


302 


0.00% 


0.96% 


0.088 


Asian 


718 


1 .25% 


416 


1 .44% 


302 


0.99% 


0.45% 


0.594 


Black 


718 


21 .45% 


416 


21 .63% 


302 


21.19% 


0.44% 


0.887 


Hispanic 


718 


5.57% 


416 


5.77% 


302 


5.30% 


0.47% 


0.786 


Multi racial 


718 


2.92% 


416 


2.40% 


302 


3.64% 


-1.24% 


0.332 


White 


718 


68.25% 


416 


67.79% 


302 


68.87% 


-1.09% 


0.758 


Gender 


















Male 


717 


38.49% 


415 


38.07% 


302 


39.07% 


-1.00% 


0.786 


Age 


635 


15.35 


371 


15.32 


264 


15.39 


-0.07 


0.086 


Exceptionality 


















Disabled/Impaired 


687 


3.78% 


409 


3.67% 


278 


3.96% 


-0.29% 


0.846 


Gifted 


701 


1 1 .98% 


410 


1 1 .95% 


291 


12.03% 


-0.08% 


0.976 


First Generation 
College 


703 


45.80% 


406 


43.84% 


297 


48.48% 


-4.64% 


0.223 


Free/Reduced Price 
Lunch 


705 


44.40% 


405 


43.95% 


300 


45.00% 


-1.05% 


0.782 


Retained 


647 


2.47% 


377 


0.80% 


270 


4.81% 


-4.02% 


0.001* 


8 m Grade Achievement 


















Math - pass 


691 


81.91% 


401 


84.79% 


290 


77.93% 


6.86% 


0.021* 


Reading - pass 


689 


97.82% 


402 


98.76% 


287 


96.52% 


2.24% 


0.047* 


Algebra 1 - pass 


182 


97.25% 


115 


96.52% 


67 


98.51% 


-1.99% 


0.432 



Notes: Statistically significant differences (at the p<0.05 level) are denoted by *. 
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Table 3: Impact of ECHS on Student Course Taking and Completion using the 
Administrative Data 





Odds-Ratio 


P-Value 


Marginal Effect 


P-Value 


Algebra 1 


Take-up 


10.971 


<0.001* 


0.136 


<0.001* 


Progress 


1.820 


0.007* 


0.102 


0.008* 


English 1 


Take-up 


1.430 


0.373 


0.008 


0.397 


Progress 


1.172 


0.530 


0.015 


0.534 


Geometry 


Take-up 


0.927 


0.742 


-0.011 


0.744 


Progress 


0.821 


0.393 


-0.013 


0.460 


College Prep. Math Course-takin 


9 1 


At least one course 


15.938 


<0.001* 


0.108 


<0.001* 


At least two courses 


2.465 


<0.001* 


0.139 


<0.001* 



Notes: Statistically significant results (at the p<0.05 level) are denoted by *. 



Table 4: Student Opinion Survey Significant 


Main Effects for ] 


Implementation Scales 




Control Group 


Treatment Group 








Scale 


Mean 


SD 


Mean 


SD 


F 


sig. 


d 


Staff-Student Relationships 


26.23 


4.00 


28.34 


3.62 


10.81 


0.00 


0.57 


Relevant Instruction 


18.63 


5.99 


20.87 


4.37 


7.56 


0.01 


0.47 


Rigorous Instruction 


35.07 


7.39 


38.76 


6.16 


11.02 


0.00 


0.57 


Student Support Activities 


26.23 


8.06 


35.36 


9.42 


41.01 


0.00 


1.00 



Notes: * df = 1,181; d = Cohen's d 
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